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Abstract 

The arrival of university ranking has changed the landscape of higher education all over the world and is likely to 
continue to influence further development nationally and internationally. This article provides an overview of 
rankings systems in which Australian universities feature and it goes on further to discuss the impact ranking 
systems have on higher education and its stakeholders. It concludes by acknowledging that ranking systems are 
viewed differently by different stakeholders and hence affect them in different ways. While no one ranking can be 
accepted as definitive, these ranking systems will remain a part of the higher education system for some time to 
come. 
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There is a new era in higher education, characterised by global competition, in which university 
ranking systems have assumed an importance. Their emergence, often controversial and subject to 
considerable debate, has been met with a lot of scepticism, some enthusiasm and an institutional unease. 
Regardless, ranking systems are here to stay and it is important to assess their effect on the higher 
education sector and its stakeholders. 
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Overview of ranking systems 

There are many standards used to assess excellence in universities but the quality of teaching and 
research is fundamental (Taylor & Braddock, n.d.). As ranking systems become a standard feature in 
higher education systems, they are also increasingly accepted as an instrument for undertaking ‘quality 
assurance’ (Sadlak, 2006). According to Harvey and Green (1993), quality is, however, relative to the user 
and circumstances in which it is applied. Ranking systems do not and cannot measure quality in higher 
education in its entirety, not least because there is still no consensus of what constitutes quality in higher 
education. Thus ranking systems incorporate the needs of some stakeholders better than others 
(Marginson, 2006a). In a review of 19 league tables and university ranking systems from around the 
world, Usher and Savino (2006) also found that ranking systems, with their use of arbitrary weightings, 
are driven by different purposes and concepts of university quality. Thus different ranking systems, 
depending on their target audience, would have different notions of quality in higher education. 

Ranking systems can be conducted either nationally or internationally, based on institutional-wide 
or sub-institutional characteristics (Usher & Savino, 2006). While there are currently three institutional 
ranking systems which compare institutions on a global basis and numerous others on a national basis, 
there are many others that focus on particular disciplines (e.g., Financial Times MBA rankings) or 
particular qualities of universities (e.g., the Journal of Black Higher Education’s Racial Diversity 
Ranking). This section provides an overview of rankings systems in which Australian universities feature 
(excluding those that focus on particular disciplines and/or programs). 

National Rankings 

Pioneered by the US News and World Report ranking in 1981, ranking systems that compare 
institutions within national boundaries are prevalent nowadays in many countries, including Canada, 

China (and Hong Kong), Germany, Italy, Poland, Spain and the United Kingdom (UK) (Usher & Savino, 
2006). Universities are also increasingly developing their own ranking systems, such as the Melbourne 
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Institute Index of the International Standing of Australian Universities (Williams & Van Dyke, 2005); and 
if one has not already been set up for a particular higher education system, it may not be surprising that 
others may develop it. The ranking of Russia’s top 100 universities is an example of one developed by a 
university outside the country, Huazhong University of Science and Technology (Sadlak, 2006). In 
Australia, there are three national ranking systems: the Learning and Teaching Performance Fund, 
Melbourne Institute Index and Good Universities Guide. 

The Learning and Teaching Performance Fund (LTPF) was announced in 2003 as part of the 
Australian government’s Our Universities: Backing Australia’s Future initiative. The purpose of the fund 
was to reward higher education providers that ‘best demonstrate excellence in learning and teaching’ 
(DEST, 2005). It is based on seven measures grouped broadly into three categories: student satisfaction, 
outcomes and success. There have been many criticisms regarding its methodology, which DEST has 
attempted to address. In 2006 (funding for 2007), a number of changes were incorporated including 
equally-weighted measures, amendments to a number of measures and outcomes being reported in four 
broad discipline groups (in contrast to the previous year’s institution-wide ranking). 

The Melbourne Institute Index aimed to take into account research performance and other areas 
such as research training and teaching (Williams & Van Dyke, 2005). It was based on 36 measures broadly 
grouped under six categories: quality/international standing of academic staff (40%), quality of graduate 
programs (16%), quality of undergraduate intake (11%), quality of undergraduate programs (14%), 
resource levels (11%) and opinions gained from surveys (8%). In 2006, the authors ranked institutions by 
discipline, using a combination of survey and performance measures that were different from previous 
years. They maintained that ‘while choice of university by undergraduates may be based heavily on the 
standing of an institution as a whole, for others, such as Ph.D. students and researchers, standing in the 
discipline is often more important’ (Williams & Van Dyke, 2006, p. 1). 
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The Good Universities Guide (GUG ) was intended as a service to prospective students and does 
not translate its outcomes into an overall university ranking. It ranked institutions across a number of 
dimensions to assist students in assessing the strengths and weaknesses of institutions. On each measure, 
institutions are grouped into five bands. A rating of five stars indicate that the institution is in the top 20% 
for that measure, four stars put it in the second 20% and so on. 

Global Rankings 

There are currently three main ranking systems which compare institutions on a global basis, the 
Shanghai Jiao Tong University (SJTU) Academic Ranking of World Universities, Times Higher 
Education Supplement (THES) World University Rankings and the Newsweek Global Universities 
Ranking. 

The SJTU ranking, first published in 2003, was developed in order to determine the gap between 
Chinese universities and world-class universities, particularly in aspects of academic or research 
performance (SJTU, 2006). The ranking of top 500 universities is predominantly based upon publication 
and citation (20% citation in leading Science and Social Science journals, 20% in articles in Science and 
Nature and 20% in the number of highly cited researchers). Another 30% is determined by alumni and 
staff with Nobel prizes and Field medals and the remaining 10% is determined by dividing the total 
derived from the above data by the number of faculty. Hence, according to SJTU, ‘quality’ in higher 
education is denoted by scientific research and Nobel Prizes. The measures do not attempt to cover aspects 
such as teaching, community building or internationalisation — it is about research excellence. For many, 
the SJTU ranking has become the ‘Achilles heel’ of universities’ reputation, with a potential to weaken 
their standing (Marginson, 2006b). 

The THES ranking began publishing their ranking tables in 2004 in recognition of the increasingly 
international nature of higher education (THES, 2006). Its ranking of the top 200 universities is 
predominantly based upon opinion surveys (40% from peers and 10% from graduate recruiters). The 
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remaining 50% includes measures of citations per faculty (20%), faculty per student ratio (10%), 
international faculty (5%) and international students (5%). The THES ranking methodology has attracted 
much criticism. Nevertheless, the THES ranking has now been in existence for the last four years (with a 
hard copy version that includes a full list of the top 500 universities also published for the first time in 
2006) and it would seem that ‘quality’ in higher education, according to the THES, is primarily about 
reputation (as measured by opinion surveys). While 50% of the index aimed to measure quality in 
teaching, research and internationalisation, quality in these areas cannot be adequately assessed using 
student-staff ratios (could be an efficiency measure), citations per faculty (which are skewed toward 
English-speaking journals in the sciences) and staff and student internationalisation measures (rewards 
quantity rather than quality). 

Published for the first time in August 2006, Newsweek claimed that the ranking table took into 
account ‘openness and diversity, as well as distinction in research’ ( Newsweek , 2006). It ranked 100 
universities and is effectively a ‘cut and paste’ of the SJTU rankings (50% from highly cited researchers;, 
articles published in Nature and Science;, articles in Science Citation Index-expanded, Social Science 
Citation Index, and Arts & Humanities Citation Index) and the THES ranking (40% from international 
faculty; international students; student/faculty score; and citations/faculty); plus an additional indicator of 
library holdings (10%). 

Web Rankings 

Recently, league tables have emerged that rank universities according to their presence on the web, 
such as G-factor International University Ranking; Webometrics Ranking of World Universities and 4 
International Colleges & Universities (4icu). The G-factor is based solely on the number of links from 
other university websites and claim that it is an objective form of ‘peer review’ because ‘the millions of 
academics, administrators and students who create the massive volume of content on university websites 
collectively vote with their feet when deciding to add a link to some content on another university website’ 
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(Hirst, 2006). Webometrics, on the other hand, use a number of indicators (size, visibility and rich files) to 
rank universities according to their web publication (Cybermetrics Research Group, 2006) while 4icu 
ranks universities in each country by web popularity as measured by a number of independent web 
metrics, including Google™ Page Rank, total number of inbound links and Alexa® Traffic Rank (4icu, 
n.d.). 


Although the ranking systems compare institutions on their web presence, each has positioned 
itself quite differently. While the G-factor and 4icu seem to target themselves toward providing 
information to prospective staff and students, Webometrics pits institutions against each other on the basis 
of their web publication and open access initiatives. 

The ranking systems described above measure quality of higher education institutions for different 
groups of stakeholders. The Organisation for Economic Co-operation and Development (OECD), in its 
policy analysis on higher education, stated that ‘governments and higher education institutions do not have 
a monopoly on the measurement of quality’ (OECD, 2006, p. 23). Ranking systems are produced by what 
each author perceives as constituting quality. Instead of trying to meet the needs of one stakeholder to the 
detriment of another, a better alternative would be a system of rankings that examine and rank institutions 
and programs based on each individual’s chosen criteria. Such a system has been developed by the Centre 
for Higher Education Development (CHE) in Germany in association with Die Zeit , which allows students 
to decide their own criteria and weightings (CHE, 2006). By providing a range of useful comparative data 
that takes account the diversity of university education, such a system could be extended for use by other 
stakeholders in higher education. 

Impact of ranking systems on higher education and its stakeholders 

There is evidence to show that ranking systems have significantly impacted on higher education 
institutions and their stakeholders, whether individually or as a group. These evidence, whether anecdotal 
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or empirical, have provided sufficient indication of how ranking systems have transformed the higher 
education landscape. 

The credibility of many institutions and senior management within these institutions has been 
affected due to the emergence of ranking systems. For instance, the University of Malaya, the oldest and 
one of the top universities in Malaysia, dropped 80 places in the THES rankings without any decline in its 
real performance due to definitional changes. This resulted in a replacement of the Vice-Chancellor and 
embarrassed the university, which claimed in an advertisement two months shy of the 2005 THES results, 
that it strived to be among the 50 best universities by 2020 (‘University of Malaya 100 years’, 2005). 

Some universities have become so concerned about rankings that they have refused to participate. 
In 1999, the University of Tokyo stated that it would no longer provide data to Asiaweek magazine for its 
annual ranking of universities in the Asia-Pacific. Also opting out were 19 mainland Chinese universities, 
including Peking University (Bacani, 1999). Asiaweek abandoned its ranking shortly thereafter. Recently, 
a group of eleven institutions in Canada indicated that they would no longer participate in Maclean’s 
magazine annual ranking of universities in that country (Birchard, 2006). Maclean’s response was that it 
would continue to rank these institutions using data from other public sources. This underlines an 
important development: with the wealth of data that is collected and made public by governments, the 
willing participation of institutions in rankings is no longer necessary. 

Rankings have influenced national governments, particularly in allocation of funding. The 
Research Assessment Exercise (RAE ) in the UK and the Performance-based Research Fund ( PBRF) in 
New Zealand were introduced in a bid to ensure that excellence in research is encouraged and rewarded. 
The LTPF is, thus far, the Australian government’s contribution to ranking systems with the Research 
Quality Framework (RQF) to come. In China, despite having repeatedly stressed that it does not support 
ranking exercises, the Chinese government has identified a group of almost 100 universities, including a 
more select group, that it believed met certain standards of excellence to receive increased funding in an 
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attempt to build a network of ‘world-class’ universities (World Education News and Reviews, 2006). 

More recently, Switzerland is also considering the introduction of an elite system, which would involve a 
boost in funding for the universities as a way to maintain their status and improve quality (Australian 
Education International, 2007). 

There is agreement among many ranking researchers and university administrators that ranking 
systems affect students’ decision-making process in selecting a higher education institution (e.g., 

Bhandari, 2006; Federkil, 2002; Filinov & Ruchkina, 2002; Vaughn, 2002). In their study, Roberts and 
Thomson (2007) found a strong correlation between league table ranking and the relative quality of 
students being admitted. They further found that applicants who seek admission to the top universities are 
more likely to use ranking tables (Roberts & Thomson, 2007). There is also anecdotal evidence that 
university recruiters are increasingly being questioned by potential students regarding the university’s 
standing in league tables. Further, in the recent Monash audit by the Australian Universities Quality 
Agency (AUQA), the Audit Panel, in discussions with students, found that Monash’s reputation and 
positions in rankings had been the deciding factor in most students’ choice of Monash as their place of 
study (AUQA, 2006). This was especially so for international students studying in Australian and overseas 
campuses and those in collaborative teaching partnerships. The increase in the number of students 
studying in universities, costs of higher education, number of students studying abroad and number of 
grants by governments and higher education institutions encouraging international student mobility have 
also increased consumer demand for information. Such information will be seen as most valuable by these 
stakeholders when presented in a manner that is easy to comprehend and when it provides advice 
(substantiated or not) regarding the best value for money (Merisotis, 2002). 

The RAE and PBRF evidently have affected staff’s decision-making process in selecting a higher 
education institution as an employer of choice (e.g., ‘Academics allege review trickery’, 2004; Grayling, 
2004; Illing, 2006). Surveys of employers in the UK and United States (US) also provide evidence that 
‘reputation of university attended’ is one of the top eight attributes that employers look for when recruiting 
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graduates (‘CollegeGrad.com releases top...’, 2006; Smith, 2006). As the global war for talent becomes 
more aggressive, an internationally recognised educational institution attached to a resume is one way of 
distinguishing oneself from another potential applicant. 

University rankings may affect the way the perception of the contribution the university makes to 
its local community, country and increasingly the world in general. More and more, the community is 
asking the question — what can a university do for us? In the US, the Washington Monthly has ranked US 
colleges based on community and national service (Washington Monthly, 2006). It claimed to be a guide 
for all Americans who are concerned whether their higher education institutions are ‘making good use of 
our tax dollars’, ‘producing graduates who can keep our nation competitive in a changing world’ and 
‘doing well by doing good’ (‘The Washington Monthly College Rankings’, 2006). 

Despite all the criticisms about methodologies used in ranking systems, they nevertheless seem to 
be shaping the behaviour of institutions (OECD, 2006). Rankings are influencing the decision-making and 
planning processes within higher education institutions. As Marginson (2007) noted, universities will 
adopt institutional policies and strategies in order to optimise their position in ranking systems. In 
developing the strategic directions of the university for instance, many institutions, directly or indirectly 
spurred by ranking systems, have now developed mission statements (or visions and goals) to become ‘one 
of the best universities in the world’ (Monash University, 2005), ‘be among the world’s truly great 
universities’ (The Ohio State University, 2007) and ‘one of the world’s top-ranked universities in the 21st 
century’ (Seoul National University, 2006). This would have a flow-on effect on other aspects of 
university operations as supporting plans and institutional policies, such as in those in finance, facilities 
and services, are developed to reinforce the strategic directions of the university. Additionally, in closing 
the quality loop, ranking results are increasingly being used as performance indicators, becoming part of 
an institution’s evaluation and monitoring processes. This, in turn, assists the development of systematic 
and practical action in order to effect improvement. 
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According to Carey (2006), rankings mean the loss of freedom and independence for institutions to 
control their brand and the terms of their success. Rankings put institutions in a mould and affect 
institutional diversity, hence affecting the way they operate within and across higher education borders. 
The Australian Education Minister, Julie Bishop, stated that the development of a diversified higher 
education sector begins with universities which differ from each other in terms of mission (Bishop, 2006). 
Similarities in universities’ mission statements above would seem to suggest that ranking systems may 
have indirectly influenced diversity (or lack of) in the higher education sector. 

If such foresight has not already been incorporated into the institution’s vision, specific areas of the 
university would have been affected by the emergence of ranking systems. For instance, the increase in 
global competition for high quality staff and costs of top quality research, coupled with the impact of 
ranking systems in staff’s decision-making process, may affect an institution’s human resource policies. 
This is especially so in the light of the RQF. A good standing in various rankings is also now reflected in 
institutions’ marketing materials. The University of Wollongong is an example, with a full page 
advertisement promoting their standing in the LTPF and GUG (Hobsons Australia, 2006, p. x). 

Conclusion 

Clearly, ranking systems measure quality for different groups of stakeholders and hence affect 
them in different ways. While the selection and choice of criteria may not meet each institution’s varying 
standards, the methodology can only improve. Ranking authors are putting their reputation at risk if the 
methodology is questionable and contains errors. This may ensure more reliable and valid systems in the 
future. A set of guidelines, Berlin Principles on Ranking of Higher Education Institutions, has also been 
established by a group of experts and producers of university rankings known as the International Ranking 
Expert Group (IREG, 2006). These include guidelines for clarity of purpose, target audience and sources 
of information. Ranking systems are also being subjected to the critical eyes of academics and education 
practitioners and there is a plethora of debates regarding their limitations, methodology and impact on the 
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higher education sector. As they become more prevalent, research into the role ranking systems, such as 
that of McDonough (Eccles, 2002), play on perceptions and behaviours of key stakeholders (such as 
potential students and their parents, employers, policy practitioners, community) will become more 
common and widespread. 

While no one ranking can be accepted as definitive, they will remain a part of the higher education 
system for some time to come because they fulfil a need — academic, commercial or otherwise. A better 
understanding of the methodology (such as the best choice of indicator measures, their validity as 
measures of effectiveness and the appropriate statistical model used to rank institutions) and effects of 
ranking systems need to inform public debate. Users of league tables should exert caution when making 
comparisons between institutions, treating results as suggestive rather than definitive. It is also the 
responsibility of ranking authors to adopt good practices and guidelines when developing ranking systems 
and to highlight the limitations about their use. More and more, institutions are being ranked and 
compared. This is best carried out based on objective information and in a spirit of collaboration. 
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